Emphasis should be placed on identifying and reporting research priorities to increase research value: An empirical analysis

Objectives To compared the presentation of research priorities in the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) clinical practice guidelines (CPGs) developed under the guidance of the GRADE working group or its two co-chair, and the Chinese CPGs. Methods This was a methodological empirical analysis. We searched PubMed, Embase, and four Chinese databases (Wanfang, VIP Database for Chinese Technical Periodicals, China National Knowledge Infrastructure and Chinese Biomedical Literature Database) and retrieved nine Chinese guideline databases or Society websites as well as GRADE Pro websites. We included all eligible GRADE CPGs and a random sample of double number of Chinese CPGs, published 2018 to 2022. The reviewers independently screened and extracted the data, and we summarized and analyzed the reporting on the research priorities in the CPGs. Results Of the 135 eligible CPGs (45 GRADE CPGs and 90 Chinese CPGs), 668, 138 research priorities were identified respectively. More than 70% of the research priorities in GRADE CPGs and Chinese CPGs had population and intervention (PI) structure. 99 (14.8%) of GRADE CPG research priorities had PIC structures, compared with only 4(2.9%) in Chinese. And 28.4% (190) GRADE CPG research priorities reflected comparisons between PICO elements, approximately double those in Chinese. The types of research priorities among GRADE CPGs and Chinese CPGs were mostly focused on the efficacy of interventions, and the type of comparative effectiveness in the GRADE research priorities was double those in Chinese. Conclusions There was still considerable room for improvement in the developing and reporting of research priorities in Chinese CPGs. Key PICO elements were inadequately presented, with more attention on intervention efficacy and insufficient consideration given to values, preferences, health equity, and feasibility. Identifying and reporting of research priorities deserves greater effort in the future.


Introduction
In the development of clinical practice guidelines (CPGs), the formation of recommendations requires a comprehensive consideration of the quality of evidence, desirable and undesirable anticipated effects, health equity and other dimensions.The quality of the evidence was heavily considered as a key determinant dimension [1,2].However, in practice, evidence retrieval, particularly of high-quality evidence, often proves challenging [3][4][5].Despite these limitations, recommendations were typically formulated based on the best available evidence.For the research gap, guideline developers tend to report research priorities through summarization and prioritization.
Simultaneously, another contrary situation arises where high-quality evidence has already existed and is robust enough that would unlikely be overturned in the future.In such cases, research saturations, a special kind of priority, should be presented to discourage further research, thus averting research waste.Furthermore, some guideline manuals also include research priorities as a form of research recommendations [6][7][8][9][10].To be more precise, research priorities are not solely evidence gaps or knowledge gaps, but rather proposals for prioritized future research based on the current evidence.
The development of guidelines requires huge intelligence and financial input.Therefore, methodological rigor is crucial, and the guideline development group is also in the most suitable position to determine the research priorities.
Since the 1990s, China has developed a considerable number of CPGs [11], and the number has been increasing rapidly over time [12].Serving as statement documents to guide Chinese clinicians in decision-making [13][14][15], CPGs has been playing a crucial and distinct role.The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approaches, developed by the GRADE working group that is led by world-leading evidencebased medicine experts, published in 2016, were considered a gold standard in guideline development, often provides a paradigm of high-quality methodology.Their structured and transparent approaches, including a fixed research priority dimension embedded in the framwork, offered valuable guidance for guideline development groups [16][17][18].
Although there were some studies on research gaps [19][20][21][22][23], the exploration of the research priorities in the guidelines was still inadequate, especially the systematic analysis of the form and content of the research priorities [24].This study systematically investigated and compared the CPGs that were developed under guidance of the GRADE working group and the Chinese CPGs, to offer reference and guidance for the development and reporting of research priorities in the future.

Identification of research priorities
We defined the research priorities as the focus of the most important future research topics addressing the needs of developing guidelines or recommendations.We usually find them in the EtD (evidence to decision) framework, discussion, evidence summary, etc., and identify their common terms including: research priorities, future research, evidence gaps, further research, etc.

Guidelines sources and searches
We searched two English databases (PubMed, Embase) and four Chinese databases (Chinese Biomedical Literature Database, China National Knowledge Infrastructure, Wanfang, and VIP Database for Chinese Technical Periodicals).We also searched the website (https://www.GRADEpro.org/)and methodological papers published by two co-chairs of the GRADE working group to find more GRADE CPGs.And we searched nine guideline databases or official websites of authoritative Chinese societies to find more Chinese guidelines (S1 File for the detailed search strategy).

Eligibility criteria
The GRADE CPGs and the Chinese CPGs that were published from the 1st January 2018 to the 31st December 2022 were included.GRADE CPGs are defined as CPGs developed under the guidance of the GRADE working group or its two co-chairs.Chinese CPGs are CPGs published in Chinese.Number of included Chinese CPGs were designed to be a random sample with twice as much as the included GRADE CPGs.Older versions and duplicate published CPGs were ineligible.

Data extraction and analysis
Two reviewers worked independently in pairs, the methods of systematic reviews of screening titles, abstracts, full text, and data extraction were strictly implemented.Unresolved differences were settled through consultation until consensus was achieved or by a third reviewer.We extracted the following information based on the standardized data extraction form designed in advance: (1) the basic characteristics of the guidelines, such as the type, scope and publication year, whether reported the research priorities, etc.; (2) the relevant information of the research recommendations, including reason, type (e.g., PICO, population, intervention, comparison, and outcome), structure, dimensions related to recommendations of research priorities, etc. Descriptive statistical analysis was used.

Registration
We have registered at INPLASY with the registration number INPLASY202350083.

Search results
The retrieval process for the GRADE CPGs and the Chinese CPGs were performed separately.For the GRADE CPGs, 523 records were retrieved from the databases, and an additional 23 records were obtained from other sources, resulting in 64 remaining records after removing duplicates and ineligible entries.Among these, four older versions and 15 records not meeting guideline criteria were excluded, leaving 45 CPGs included.Regarding Chinese CPGs, 225782 and 201 records were retrieved from databases and other sources respectively, 595 remained after removing duplicate and unqualified records.3 older versions, 27 duplicate publications, 22 not guidelines were excluded.Finally, 543 Chinese CPGs were identified, of which 90 were randomly selected for inclusion (S2 File and Fig 1).

Guideline characteristics
A total of 135 CPGs were included (45 GRADE CPGs and 90 Chinese CPGs).Among the GRADE CPGs, about 90% (40, 88.9%) reported research priorities, with more than half (22, 55%) orienting to the whole guidelines.In Chinese CPGs, only less than 30% (26, 28.9%) of the guidelines reported research priorities, of which more than 70% (20, 76.9%) were for the whole guidelines.Furthermore, ten (25.0%) of the GRADE CPGs reported more than 20 research priorities, while the Chinese CPGs all presented fewer than 20 (Table 1).

Structure and presentation form of research priorities in CPGs
The GRADE CPGs reported a total of 668 research priorities, while the Chinese CPGs reported only 138.As for the reasons for developing the research priorities, 515 (77.1%)GRADE CPG research priorities reported the lack of evidence, while the Chinese CPG research priorities further reported the lack of high-quality evidence (71, 51.4%).Analyzing the structure of research priorities, we found that over 70% of both GRADE CPGs and Chinese CPGs had structures (P, I), comprising 475 (71.1%) and 101 (73.2%) respectively.In contrast, there were 99 (14.8%)GRADE research priorities with PIC structures compared to only 4 (2.9%) in the Chinese CPGs.Notably, 190 (28.4%)GRADE CPG research priorities involved the comparison between PICO elements, approximately double those in Chinese CPG research priorities (Table 2).
We identified 1076 and 128 descriptions related to the research priorities in the GRADE CPGs and Chinese CPGs, respectively.The descriptions of the research priorities in the GRADE CPGs were concentrated, with approximately 60% (635, 59.0%) described as "research priorities/needs/agenda/are needed/question", with "research priorities" alone accounting for 51.4% (553).In contrast, descriptions of Chinese research priorities were more scattered and relatively simple.The maximum number of "questions to be solved/studied/ addressed" described was 43 (33.6%),followed by "further research / studies / clarified" and

Focus of research priorities
Both the research priorities of the GRADE CPGs and the Chinese CPGs were clustered around efficacy of interventions, accounting for 42.1% (281) and 52.2% (72), respectively.Regarding the proportion of research priorities for comparative effectiveness, that is the condition if different interventions would need to be compared to address the research priority, GRADE research priorities were approximately double those in Chinese (accounted for 30.2% versus 15.2%).More than half of the research priorities in the GRADE CPGs and the Chinese CPGs reflected the dimension of desirable or undesirable anticipated effects, which are in the GRADE EtD framework for formulation of recommendations.The GRADE CPG research priorities considered a higher proportion of health equity and acceptability, while the Chinese is more concerned about the feasibility and quality of evidence (Table 3).

Recommendations for the study methods and details of intervention implementation in the research priorities
The number of research priorities concerning recommended research methods among GRADE CPGs and Chinese CPGs was relatively small, with randomized controlled trials being the primary research method, accounting for 6.4% (43) and 8.0% (11), respectively.GRADE CPGs also recommends observational studies (14.2.1%) and risk assessment model

Principal findings
This  different: the former was more focused on health equity and acceptability, while the latter was more concerned on feasibility and quality of evidence.

Strength and limitations
In our study, we conducted detailed analysis of the presentation form, structure and type of research priorities in GRADE CPGs and Chinese CPGs.To our knowledge, no comprehensive and systematic research of such research priorities before.There were limitations in our study.We accepted and analyzed the research priorities reported in the CPGs without further investigating the appropriateness of the underlying logic of the prioritization.Additionally, in most cases, the presented research recommendations were due to the lack of evidence or the low-quality of evidence.Different guideline development groups may have differences or confusion about the definition of the lack of evidence and quality of evidence.

Relation to previous work
A previous study formulated research priorities by developing a process, benefiting from considering the evidence base while identifying current knowledge gaps as well as any uncertainty [25].Other studies focused on the structure of the research priorities, which summarized EPI-COT (evidence, population, intervention, comparison, outcome, and time) as an essential structural element and emphasized the burden of disease and type of study as details that should be further considered [26][27][28].Robinson and his colleagues identified 62 research gaps in five guidelines published by the Cystic Fibrosis Foundation and found that only 20% of the evidence gap were identified as research priorities by the guideline development group.They suggested that guideline developers should articulate research priorities more explicitly and systematically [24].Although these studies explored research gaps or research priorities, they often conflated the two, which may not accurately reflect the connotation of the research priorities.

Implications for research and guidelines
We emphasize the distinction between evidence gaps and research priorities as an independent dimension.Not all evidence gaps indicate a reason to be prioritized.Significant effort in enhancing the methods and standards of identifying and reporting of research priorities should be taken.
First of all, the key structure elements of research priorities should be reported as comprehensive and standardized as possible.Clinical question formulation frameworks such as the PICO or EPICOT models can facilitate the transformation of research priorities into clinical questions efficiently and accurately, ensuring consistency with the actual needs of clinical practice and future research.Secondly, attention should be directed towards addressing research priorities within the dimensions of the GRADE EtD framework, including health equity, feasibility, values and preferences.Thirdly, guideline development groups should be encouraged to standardize the description of research priorities in guideline reporting, positioning them consistently and independently whenever feasible to aid identification by guideline users.Lastly, more attention should be paid to identify research priorities on details of intervention implementation, which are essential for developing really practical clinical recommendations.Besides the above, another issue needs to be clarified.The rationale for proposing research priorities should not be solely tied to evidence absence or low-quality.It would be of great value of we could point out research saturation areas to avoid research waste when there is high-quality evidence that is robust enough that very unlikely to be overturned in the future.

Conclusion
There was still considerable room for improvement in the developing and reporting of research priorities in Chinese CPGs.Key PICO elements were inadequately presented, with more attention on intervention efficacy and insufficient consideration given to values, preferences, health equity, and feasibility.Identifying and reporting of research priorities deserves greater effort in the future.

Table 1 . Characteristics of included clinical practice guidelines N %.
, clinical practice guidelines; GRADE, Grading of Recommendations Assessment, Development and Evaluation; Chinese, Traditional Chinese Medicine.
a Some guidelines present research priorities at multiple places, so the total number of items is not 45, 90, 135.b Research recommendations as independent recommendation.CPG

Table 2 . Structure and presentation form of research priorities N %.
CPG, clinical practice guidelines; GRADE, Grading of Recommendations Assessment, Development and Evaluation; Chinese, Traditional Chinese Medicine; PICO, population, intervention, comparison, outcome.
(26)y reviewed 45 GRADE CPGs and 90 Chinese CPGs published in 2018 to 2022, and identified 668 and 138 research priorities, respectively.While 88.9% (40) of GRADE CPGs reported research priorities, only 28.9%(26)of Chinese CPGs did so.Analysis of the PICO elements of research priorities revealed that about 70% of both GRADE and Chinese research priorities had a PI structure.Furthermore, 14.8% (99) of GRADE CPG research priorities had PIC structures, compared to only 2.9% (4) in Chinese.Of notice, 28.4% (190) of GRADE CPG research priorities reflected comparisons between PICO elements, approximately double the proportion observed in Chinese research priorities.More than half of both GRADE and Chinese CPG research priorities focused on the dimensions of desirable or undesirable anticipated effects of the GRADE EtD framework.However, for the other dimensions, the concerns were

Table 3 . Focus of research priorities in GRADE CPGs and Chinese CPGs N %.
We classified a research priority as comparative effectiveness if different interventions or population would need to be compared to address the research priority.More treatment or studies are needed b More treatment or studies are needed c Basic science research, intervention-standardized definition, decision aids, policy-making, specification.d Sometimes a research priority may focus on multiple dimensions of recommendations, so the total number of items does not equal 668,138,806.CPG, clinical practice guidelines; GRADE, Grading of Recommendations Assessment, Development and Evaluation; Chinese, Traditional Chinese Medicine. a